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This letter proposes a novel adaptive reduced-rank filtering scheme based on joint iterative optimization of 
adaptive filters. The novel scheme consists of a joint iterative optimization of a bank of full-rank adaptive filters 

o ' 

that forms the projection matrix and an adaptive reduced-rank filter that operates at the output of the bank of filters. 
We describe minimum mean-squared error (MMSE) expressions for the design of the projection matrix and the 
reduced-rank filter and low-complexity normalized least-mean squares (NLMS) adaptive algorithms for its efficient 
implementation. Simulations for an interference suppression application show that the proposed scheme outperforms 
in convergence and tracking the state-of-the-art reduced-rank schemes at significantly lower complexity. 
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I. Introduction 



In adaptive filtering [1], one can find a huge number of algorithms with different trade-offs between performance 
.Cy. 1 and complexity. They range from the simple and low-complexity least-mean squares (LMS) algorithms to the fast 
converging though complex recursive least squares (RLS) techniques. Several attempts to provide cost-effective 
adaptive filters with fast convergence performance have been made with variable step-size algorithms, data-reusing, 
sub-band and frequency-domain schemes and RLS algorithms with linear complexity. A challenging problem which 
remains unsolved by conventional techniques is that when the number of elements in the filter is large, the algorithm 
requires a large number of samples to reach its steady-state behavior. In these situations, even RLS algorithms require 
an amount of data proportional to 2M [1] in stationary environments to reach steady state, where M is the filter 
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length, and this may lead to unacceptable convergence performance. In dynamic scenarios, large filters usually fail 
or provide poor performance in tracking signals embedded in interference. 

Reduced-rank filtering [2]-[9| is a powerful and effective technique in low sample support situations and in 
problems with large filters. The advantages of reduced-rank adaptive filters are their faster convergence speed and 
better tracking performance than full-rank techniques when dealing with large number of weights. Several reduced- 
rank methods and systems have been proposed in the last several years, namely, eigen-decomposition techniques 
(3) -El, tne multistage Wiener filter (MWF) 0, Q and the auxiliary vector filtering (AVF) algorithm [8 |. The 
main problem with the above techniques is their high complexity and the existence of numerical problems for 
implementation. 

In this work we propose an adaptive reduced-rank filtering scheme based on combinations of adaptive filters. 
Unlike related work on combinations of full-rank filters iTTOl . the novel scheme consists of a joint iterative 
optimization of a bank of full-rank adaptive filters which constitutes the projection matrix and an adaptive reduced- 
rank filter that operates at the output of the bank of full-rank filters. Differently from [flTI. the proposed scheme 
estimates a scalar, allows filter updates for each successive observation, is adaptive and has low complexity. The 
essence of the proposed approach is to change the role of adaptive filters. The bank of adaptive filters is responsible 
for performing dimensionality reduction, whereas the reduced-rank filter effectively estimates the desired signal. 
Despite the large dimensionality of the projection matrix and its associated slow learning behavior, the proposed 
and existing Q, J8 ] reduced-rank techniques enjoy in practice a very fast convergence. The reason is that even an 
inaccurate or rough estimation of the projection matrix is able to provide an appropriate dimensionality reduction 
for the reduced-rank filter, whose behavior will govern most of the performance of the overall scheme. We describe 
MMSE expressions for the design of the projection matrix and the reduced-rank filter along with simple NLMS 
adaptive algorithms for its computationally efficient implementation. The performance of the proposed scheme is 
assessed via simulations for CDMA interference suppression. 

II. Reduced-Rank MMSE Parameter Estimation and Problem Statement 
The MMSE filter is the vector w = [w 1 w 2 ... %] T , which is designed to minimize the MSE cost function 

J = E[\d(i)-w H v(i)\ 2 ] (1) 

where d(i) is the desired signal, r(i) = [r® . . . r^j_ 1 ] T is the received data, (-) T and (-) H denote transpose 
and Hermitian transpose, respectively, and E[-] stands for expectation. The set of parameters w can be estimated 
via standard stochastic gradient or least-squares estimation techniques |T). However, the laws that govern the 
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convergence behavior of these estimation techniques imply that the convergence speed of these algorithms is 
proportional to M, the number of elements in the estimator. Thus, large M implies slow convergence. A reduced- 
rank algorithm attempts to circumvent this limitation in terms of speed of convergence by reducing the number of 
adaptive coefficients and extracting the most important features of the processed data. This dimensionality reduction 
is accomplished by projecting the received vectors onto a lower dimensional subspace. Specifically, consider an 
M x D projection matrix Sd which carries out a dimensionality reduction on the received data as given by 

r(i) = Sgr(i) (2) 

where, in what follows, all D-dimensional quantities are denoted with a "bar". The resulting projected received 
vector r(i) is the input to a tapped-delay line filter represented by the D vector w = [w 1 w 2 ... w D ] T for time 
interval i. The filter output corresponding to the ith time instant is 

x(i) = w H f(i) (3) 

If we consider the MMSE design in (1) with the reduced-rank parameters we obtain 

w = R _1 p (4) 

where R = E{r(i)r H (i)] = S^RSd is the reduced-rank covariance matrix, R = E{r(i)r H (i)] is the full-rank 
covariance matrix, p = E[d*(i)f(i)} = Sgp and p = E[d*(i)r(i)]. The associated MMSE for a rank D estimator 
is expressed by 

MMSE = a\- p^R" 1 ? = a\- p // S D (SgRS D )~ 1 Sgp (5) 

where a\ is the variance of d{%). In the Appendix, we provide a necessary and sufficient condition for a projection 
Sd with dimensions M x D to not modify the MMSE and discuss the existence of multiple solutions. Based 
upon the problem statement above, the rationale for reduced-rank schemes can be simply put as follows. How to 
efficiently (or optimally) design a transformation matrix Sd with dimensions M x D that projects the observed 
data vector r(i) with dimensions M x 1 onto a reduced-rank data vector r(i) with dimensions D x 1? In the next 
section we present the proposed reduced-rank approach. 

III. Proposed Reduced-Rank Scheme 

Here we detail the principles of the proposed reduced-rank scheme using a projection operator based on adaptive 
filters. The novel scheme, depicted in Fig. 1, employs a projection matrix So(i) with dimensions MxDto process 
a data vector with dimensions Mxl, that is responsible for the dimensionality reduction. The reduced-rank filter 



w(i) with dimensions Dxl processes the reduced-rank data vector f (i) in order to yield a scalar estimate x(i). The 
projection matrix So{i) and the reduced-rank filter w(i) are jointly optimized in the proposed scheme according 
to the MMSE criterion. 
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Fig. 1. Proposed Reduced-Rank Scheme. 

Specifically, the projection matrix is structured as a bank of D full-rank filters Sd(i) = [si,d{i) S2,<zW • • • s M,d(i)] T 
(d = 1, . . . , D) with dimensions Mxl as given by Sd(«) = [ Si(i) | S2(i) \ ... |sd(«) ]. Let us now mathematically 
express the output estimate of the reduced-rank scheme as a function of the received data r(i), the projection 
matrix So(i) and the reduced-rank filter w(i): 

x(i) = w H (i)Sg(i)r(i) = w H (i)r{i) (6) 

Note that for D = 1, the novel scheme becomes a conventional full-rank filtering scheme with an addition weight 
parameter wd that provides a gain. For D > 1, the signal processing tasks are changed and the full-rank filters 
compute a subspace projection and the reduced-rank filter estimates the desired signal. 

The MMSE expressions for the filters So(i) and w(z) can be computed through the following cost function: 

J = E[\d(i)-w H (i)S%(i)r(^ 2 

= E[\d(i)-w H (i)f(i)\ 

By fixing the projection and minimizing (7) with respect to w(i), the reduced-rank filter weight vector 

becomes 

w(i) =R.- 1 (i)p(i) (8) 

where R(i) = E[S%(i)r(i)r H (i)S D (i)] = E[f (i)f H (»)], p(i) = £[d*(i)Sg(i)r(i)] = E[d*(i)r(i)]. We proceed 
with the proposed joint optimization by fixing w(i) and minimizing (7) with respect to So(i). We then arrive at 
the following expression for the projection operator 

S D (i) = K~ 1 (i)P D (i)R w (i) (9) 

where R(i) = £[r(i)r H (i)], P D (i) = E[d*(i)r(i)w H (i)] and R w (i) = E[w(i)w H (i)]. The associated MMSE is 

MMSE = ^-p H (i)R" 1 (i)p(i) (10) 
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where a\ = E[\d(i)\ 2 ]. Note that the filter expressions in (8) and (9) are not closed-form solutions for w(i) and 
Sn(i) since (8) is a function of So(i) and (9) depends on w(i) and thus it is necessary to iterate (8) and (9) 
with an initial guess to obtain a solution. The MWF |6] employs the operator Sd = [p Rp • • • R/ 0-1 ?] that 
projects the data onto the Krylov subspace. Unlike the MWF approach, the new scheme provides an iterative 
exchange of information between the reduced-rank filter and the projection matrix and leads to a much simpler 
adaptive implementation than the MWF. The projection matrix reduces the dimension of the input data, whereas 
the reduced-rank filter attempts to estimate the desired signal. The key strategy lies in the joint optimization of the 
filters. The rank D must be set by the designer to ensure appropriate performance and the reader is referred to 11121 
for rank selection methods. In the next section, we seek iterative solutions via adaptive algorithms. 

IV. Adaptive Algorithms 

Here we describe an adaptive NLMS implementation, convergence conditions and detail the computational 
complexity in arithmetic operations of the proposed reduced-rank scheme. 

A. Adaptive Algorithms 

Let us consider the following Lagrangian cost function 

C = \\w(i + 1) - w(i)\\ 2 + \\S D (i + 1) - S D (i)\\ 2 

+ ®[\l(d(i) - w H (i + l)Sg(i)r(i)] (11) 

+ M[\Ud(i)-w H (i)S%(i + l)r(i)], 
where Ai, A2 are scalar Lagrange multipliers, || • || 2 denotes the Frobenius norm and the operator retains the 
real part of the argument. By computing the gradient terms of (11) with respect to w(i + 1), Sd(« + 1), Ai and 
A2, setting them to and solving the resulting equations, we obtain: 

Vw( I+ i)£ = 2(w(i + 1) - w(i)) + Sg(i)r(i)Ai = (12) 

V SD(i+1) £ = 2(S D (i + 1) - Sjj(i)) + r(i)w H (i)X 2 = (13) 

V Al £ = d(i) - w H (i + l)Sg(i)r(i) = (14) 

Va 2 £ = d(i) - w H (i)S%{i + l)r(t) = (15) 

By solving the above equations and introducing the convergence factors hq and 770, the proposed jointly optimized 
and iterative NLMS algorithms for parameter estimation become 

w(t + l)=w(i) + ^(i)e*(t)r(*), (16) 



S D (i + 1) = S D (i) + 7?(i)e*(i)r(i)w H (i), (17) 

where e(i) = d(i) — w H (i)Sp(i)r(i), fi(i) = [io/(r H (i)r(i)) and = r]Q / (~w H (i)w(i)r H (i)r(i)) are the 
time-varying step sizes. The algorithms described in (16)-(17) have a complexity O(DM). The proposed scheme 
trades-off a full-rank filter against D full-rank adaptive filters as the projection matrix S/)(i) and one reduced-rank 
adaptive filter w(i) operating simultaneously and exchanging information. The iteration and convergence occurs 
over several observations and here we consider only one iteration per symbol (i). 



B. Convergence Conditions 

Define the error matrices at time index i as e^(i) = w(i) — w opt and es D (i) = Sd(z) — S^opt, where w opt and 
SD,opt are the optimal parameter estimators. Because of the joint optimization procedure, both filters have to be 
considered jointly. By substituting the expressions of e^(i) and es D (i) in (16) and (17), taking expectations and 
simplifying the terms, we obtain 

E[e^(i + 1)] 
E[e Sn (i + l)\ 







= A 




_ E[e SD (i)\ _ 



+ B 



(18) 



where A 



(I-£[/i(i)]R) 







E[v(i)]aiRS Dt0pt (I - E[u(i)]alR) 



, B 



E[fM(i))(KS D (i)w opt -p) 
E[v(i)]al(RS Di0pt -w op t - p) 



and cT^ 



i?[||w(i)|| 2 ]. The above equation implies that the stability of the algorithms depends on the spectral radius of A. 
For convergence, the step sizes should be chosen such that the eigenvalues of A H A are less than one. 



C. Computational Complexity 

Here, we detail the computational complexity in terms of additions and multiplications of the proposed schemes 
with NLMS and other existing algorithms, namely the Full-rank with NLMS and RLS, the MWF Q with NLMS 
and RLS and the AVF |8], as shown in Table 1. The MWF |7] has a complexity 0{DM 2 ), where the variable 
dimension of the vectors M = M — d varies according to the rank d = 1,...,D. The proposed scheme is much 
simpler than the Full-rank with RLS, the MWF and the AVF and slightly more complex than the Full-rank with 
NLMS (for D « M, as will be explained later). 

V. Simulations 

In this section we assess the proposed reduced-rank scheme and algorithms in a CDMA interference suppression 
application. We consider the uplink of a symbol synchronous BPSK DS-CDMA system with K users, N chips 
per symbol and L propagation paths. Assuming that the channel is constant during each symbol interval and the 
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TABLE I 

Computational complexity of algorithms. 

Number of operations per symbol 



Algorithm 

Full-rank-NLMS 

Full-rank-RLS 

Proposed-NLMS 

MWF-NLMS 

MWF-RLS 

AVF 



Additions 

3M- 1 
3(M - l) 2 + M 2 + 2M 
2DM + M + 4.D — 2 
D(2M 2 - 3M + 1) 
D(4(M - l) 2 + 2M) 
£>(M 2 + 3(M - l) 2 ) - 1 
+D(5(M - 1) + 1) + 2M 



Multiplications 

3M + 2 
6M 2 + 2M + 2 
3 -DM + M + 3D + 6 
D(2M 2 + 5M + 7) 
L>(4M 2 + 2M + 3) 
D(4M 2 + 4M + 1) 
+4M + 2 



randomly generated spreading codes are repeated from symbol to symbol, the received signal after filtering by a 
chip-pulse matched filter and sampled at chip rate yields the M-dimensional received vector 

K 

r(i) = J2Hk(j>)A k C k b k (j,)+n(i), (19) 

k=l 

where M = N + L — l, n(i) = [n\{i) . . . n.M{i)] T is the complex Gaussian noise vector with E[n(i)n H (i)] = a 2 I, 
the symbol vector is h k (i) = [b k (i + L s — 1) ... b k (i) ... b k (i — L s + the amplitude of user k is A k , L s is 
the intersymbol interference span, the ((2L S — 1) • N) x (2L S — 1) block diagonal matrix C k is formed with TV-chips 
shifted versions of the signature = [<Xfc(l) . . . ak{N)] T of user k and the M x (2 • L s — 1) • N convolution matrix 
Hfc(i) is constructed with shifted versions of the L x 1 channel vector h k (i) = [h kj o(i) . . . hk,L p -i{i)] T on eacn 
column and zeros elsewhere. For all simulations, we use w(0) = Od,i, Sd(0) = [Id Qd,m-d] T , assume L = 9 
as an upper bound, use 3-path channels with relative powers given by 0, —3 and — 6 dB, where in each run the 
spacing between paths is obtained from a discrete uniform random variable between 1 and 2 chips and average 
the experiments over 200 runs. The system has a power distribution amongst the users for each run that follows a 
log-normal distribution with associated standard deviation equal to 1.5 dB. 

We compare the proposed scheme with the Full-rank [1], the MWF Q and the AVF (H techniques for the 
design of linear receivers, where the reduced-rank filter w(i) with D coefficients provides an estimate of the desired 
symbol for the desired user (user 1 in all experiments) using the signal-to-interference-plus-noise ratio (SINR) Q. 
We consider the SINR performance versus the rank D with optimized parameters (^o> an d forgetting factors A) 
for all schemes. The results in Fig. 2 indicate that the best rank for the proposed scheme is D = 4 (which will be 
used in the remaining experiments) and it is very close to the optimal full-rank MMSE. Studies with systems with 
different processing gains show that D is invariant to the system size, which brings considerable computational 
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Fig. 2. SINR performance versus rank (D). 



savings. In practice, the rank D can be adapted in order to obtain fast convergence and ensure good steady state 
performance and tracking after convergence. 

We show an experiment in Fig. 3 where the adaptive filters are set to converge to the same SINR. The NLMS 
version of the MWF is known to have problems in these situations since it does not tridiagonalize its covariance 
matrix [7 1 and thus is unable to approach the MMSE. The curves show an excellent performance for the proposed 
scheme and algorithms, which converge much faster than the full-rank filter, are comparable to the more complex 
MWF-RLS and AVF schemes, at much lower complexity. 

The BER convergence performance in a mobile communications situation is shown in Fig. 4. The channel 
coefficients are obtained with Clarke's model |[T3l and the adaptive filters of all methods are trained with 250 
symbols and then switch to decision-directed mode. The results show that the proposed scheme has a much better 
performance than the existing approaches and is able to adequately track the desired signal. 

VI. Conclusions 

We proposed a novel reduced-rank scheme based on joint iterative optimization of adaptive filters with a low 
complexity implementation using NLMS algorithms. In the proposed scheme, the full-rank adaptive filters are 
responsible for estimating the subspace projection rather than the desired signal, which is estimated by a small 
reduced-rank filter. The results for CDMA interference suppression show a performance significantly better than 
existing schemes and close to the optimal full-rank MMSE. 
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Fig. 3. SINR performance versus number of received symbols. 
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Fig. 4. BER performance versus number of received symbols. 



Appendix 

Given a M x D projection matrix S^, where D < M, the MMSE is achieved if and only if w which minimizes 
(1) belongs to the Range{Sz)}, i.e. w lies in the subspace generated by Sd- In this case, we have MMSE(w) = 
MMSE(w) = aj- p^R x p. For a general S D , we have MMSE(w) > aj - p^R x p. From the above analysis, 
we can conclude that there exists multiple solutions to the proposed optimization problem. However, our studies 
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indicate that there are no local minima and the performance is insensitive to initialization, provided we select the 
initial values w(0) and Sd(0) which do not instabilize the algorithm and annihilate the signal, respectively. 
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